Design - Page Identity

October 3, 2026

 

 

The technote mentioned above explains why OneNote IDs are fragile and how that was measured. This document explains what OneMore does about it. This is planned future work

 

1. Problem and scope

 

OneNote regenerates every page, section, and notebook ID when a notebook is closed and reopened, and a moved page receives a new page ID. Created time, title, and content survive. OneMore stores those IDs in its database and later treats them as keys, so references silently stop resolving.

 

What breaks today

 

 

There is a manual repair path, TargetChecker (the Check button in the manage dialogs). It resolves by ID and then by location path, but it is manual, matches by title path only, and its repairs are never saved: UpdateFavorite and UpdateWindow do not write IDs or the URI, so a repaired reference reverts when the dialog closes.

 

Goals

 

1. A reference to a page, section, section group, or notebook keeps resolving across close/reopen, page moves (between sections and between notebooks), and creation-date edits.
2. Nothing is written to the user's pages to identify them - hence does not touch the last-update-time.
3. One shared mechanism serves hashtags, favorites, and layouts, rather than three private ones.
4. References that cannot be resolved are shown as broken, never silently dropped or deleted.

 

Non-goals

 

• The full-text search index (planned later; it will consume this layer).
• Bookmarks and reminders. Both store OneNote IDs and are natural future consumers, but are out of scope here.
• Roaming or sharing OneMore.db between machines. Each machine keeps its own database and its own keys.
• Paragraph (object) IDs. Their stability across a reopen has not been measured.

 

 

2. Concepts

 

Page key

A page key is an integer assigned by OneMore in its own database (identity_page.pageKey, AUTOINCREMENT). It is not derived from the page, so OneNote cannot change it. What can fail is attaching the key to the right page again after OneNote changes the IDs; that is the job of the matcher.

 

Keys are never reused after a record is deleted. A reused key would attach data stored against the old page to a different page.

 

The page key is a signed 64-bit integeger, so has a range of 1 to 263-1

 

Container keys

Notebook and section IDs also change, so containers are identified by name and path instead of a surrogate:

 

 

Fingerprint

What is known about a page from the hierarchy alone, without loading it: notebookKey, sectionKey, title, created (millisecond precision), and modified. modified is used only to pair pages that are otherwise identical.

 

Resolution ladder

Each current page that has no stored record with the same OneNote ID is compared against the stored records that no page has claimed, strongest evidence first. Each step considers only what earlier steps left unclaimed.

 

 

Measured results for steps 2 to 5 are in the technote.

 

A further step that loads the page and matches a hash of its text was considered and deliberately left out of this design. Nothing in hashtags, favorites, or layouts needs it; only the later text index would. It will be added with the index, as a small upgrade of the identity catalog (section 7). When a page matches nothing above, it is treated as a new page, which costs a re-read and is never wrong.

 

 

Rules

• One-to-one only. A step matches a page to a record only when exactly one candidate qualifies on each side. If several pages are indistinguishable, the match is declined and they are treated as new. That costs a re-read, but is never wrong. The single exception is identical pages in the same location with the same modified time: their content is the same, so which record each gets does not matter.
• Claimed records are excluded. A page copied into another notebook keeps its title and creation time. Because the original claims its own record by ID first, the copy cannot be mistaken for it.
• A weak match (step 5) forces a re-read of the page by any consumer that caches content, because title alone is not proof.
• Do not delete on the first miss. A page that is not seen is marked missing (missingSince) and kept. It is deleted only after a grace period (hours), because a reopened notebook fills in over minutes. If it reappears under a new ID, the ladder reconnects it.
• Skipped sections are not deleted sections. A locked or unreadable section is recorded as skipped for that pass, so its pages are left alone instead of being marked missing.
• Refresh created on every pass. A creation-date edit changes neither the page ID nor lastModifiedTime, so the stored value must be refreshed from the hierarchy every time.

 

What is deliberately not part of identity

 

• No marker written into pages. The earlier omPageID stamp worked across reopens but is a write to a page the user did not edit.
• No reliance on OneNote IDs beyond one session. They are used to load or navigate now, never remembered.

 

 

3. Architecture

 

 

Components

 

 

Why a separate, always-on identity pass

Favorites and layouts can point at pages in any notebook, including ones the user excluded from hashtag scanning, or when hashtag scanning is disabled. If identity were produced only by the hashtag scan, those references would have nothing to resolve against. The pass is cheap because it loads no page content.

 

Pipeline order for each cycle: identity pass, then hashtag stage, then workspace healing.

 

Decision: one pipeline in one background loop. The three stages run in that order every cycle, so a hashtag scan can never run against identity that has not caught up. The hashtag service's disabled setting skips only the hashtag stage; the identity pass and healing keep running, because favorites and layouts depend on them. The tray's scheduled scan and rebuild call the same pipeline, so they too begin with an identity pass. The pipeline's tuning values, such as the missing-page grace period, are named constants.

 

 

4. Data model

 

 

Notes:

 

• The OneNote IDs that remain in favorite and layout_window are last-known values, kept current by the healer. They are a cache, not the identity.
• layout_window.pageID stays NOT NULL, so a newly captured window always stores the ID it had when saved.
• Notebook, section-group, and section favorites carry no pageKey. They store notebookKey and sectionKey and are re-resolved by name and path.
• The hashtag tables keep the moreID column name and simply hold the page key as text. Renaming it would require a table rebuild for no behavioral gain.
• There is deliberately no contentHash column. The later text index will add one (with the matching step) as an identity 1 to 2 upgrade.

 

 

5. Behavior

 

5.1 Background identity pass

 

 

A notebook whose hierarchy could not be read is left out of the scope of the pass, so a failed read is never mistaken for an empty notebook.

 

5.2 A notebook is reopened

 

 

5.3 Navigating a favorite

 

 

5.4 Restoring a layout

 

 

5.5 Adding a favorite or saving a layout

 

The page's fingerprint is captured at creation time, so the reference is born with a key.

 

 

5.6 Background healing

 

 

 

6. Applying it to each feature

 

6.1 Hashtags

 

• Tags are keyed by the page key, stored as text in the existing moreID columns. ReadPageTags and WriteTags use the key; hashtag_page is joined by key, not by pageID.
• The scanner stops matching by ID and stops writing omPageID into pages (the stamp set by HashtagPageScanner.SetMoreID and written by HashtagScanner.ScanPage, and the stamp in DuplicatePageCommand). It keeps its one legitimate page write, which applies the hashtag style the user chose.
• Notebook rows are adopted by name after a reopen: a stored notebook whose ID is gone is taken to be the open notebook of the same name, keeping its inclusion setting and last scan. If either duplicate was excluded, the notebook stays excluded. This also removes the 100-page "new notebook" threshold problem for reopened notebooks.
• Which pages are read: changed pages, new pages, weak matches, and pages that have tags and whose ID changed (the object IDs stored with their tags are probably stale). Reopening a notebook therefore re-reads only its tagged pages, not the whole notebook.
• Stored page info (pageID, sectionID, notebookID, path, name) is refreshed from the hierarchy for every tagged page on every pass, because the dialogs compare these with the IDs of the notebooks open now.
• Deleting tags is tied to purging missing pages after the grace period, replacing per-section phantom deletion.
• One-time migration: tags recorded under the old stamp keys are cleared and the scan time reset, so one background scan rebuilds them. Tags are derived from page text, so nothing is lost; notebook selections are kept.
• HashtagCommand finds the current page's key by asking the resolver for its page ID, instead of reading a stamp from the page.

 

6.2 Favorites

 

• Page favorites gain pageKey. Navigation resolves the key to current IDs and a freshly generated URI (section 5.3).
• Section, section-group, and notebook favorites gain notebookKey and sectionKey and are re-resolved by name and path, using the same keys the identity layer already builds. They need no surrogate and no scan, only a resolve step. A notebook favorite today stores the notebook ID in sectionID and uri; the key columns replace that dependence.
• Unique indexes. idx_favorite_target_page (on pageID) and idx_favorite_target_section (on sectionID) are partial unique indexes and cannot be altered, so they are dropped and recreated on the keys inside the upgrade transaction.
• Legacy rows (created before keys existed) are backfilled lazily by the healer, using TargetChecker's existing logic: match the stored IDs against the current hierarchy, else walk location by name. Healed values are then persisted, which today they are not.
• The existing root-folder convention (folderID = 0, not NULL) is left alone.
• The manual Check remains as a fallback and now saves what it repairs.

 

6.3 Layouts

• layout_window gains pageKey.
• RestoreLayoutCommand compares open windows against the resolved page ID, and navigates by the resolved URI.
• A window that cannot be resolved is reported, not silently skipped.
• LayoutsProvider has no version table today. It gains layouts_schema, following exactly what favorites did when it introduced favorites_schema.

 

6.4 Export and import compatibility

Favorites and layouts can be exported to JSON and imported again, possibly after the database has been upgraded, or on another machine. The existing code is already tolerant, which is what this design relies on:

 

• Export serializes the model objects with Newtonsoft (ExportFavoritesCommand, ExportLayoutsCommand). Import deserializes with default settings, so properties missing from the file take their defaults and unknown properties are ignored. It then writes each row through WriteFavorite / WriteWindow, which use an explicit column list rather than the old schema's shape. Importing resets every database-local ID first (favorite.ID = 0, FolderID and LayoutID reassigned).

 

What the design must therefore guarantee:

 

1. An old file imports into an upgraded database. Old files have no pageKey, notebookKey, or sectionKey, so those arrive unset. The new WriteFavorite / WriteWindow must accept that and store NULL. The model property for pageKey must be nullable: a plain integer would deserialize as 0 and be stored as a real key.
2. pageKey is database-local, like favorite.ID, and is never trusted from a file. A key is only meaningful in the database that issued it. Key 17 on another machine, or in a database that was since reset, is a different page. Export therefore omits it, and import discards any value it finds, exactly as it already discards ID. Imported rows are resolved afterwards by the healer.
3. Imported rows are treated as legacy rows. The healer backfills them the same way as rows that predate the upgrade: stored IDs first (valid only if they happen to match this machine's current IDs), then location path plus title. A file from another machine will mostly resolve by the location fallback, with today's limits: no created time, so duplicate titles can be ambiguous, and unresolved rows are shown as broken.
4. Duplicate detection must not depend on pageKey alone. The unique indexes move to the keys, but imported rows have none yet, so a re-import could create duplicates that the old pageID index used to reject. The import path needs its own duplicate check (on location and title, or on the stored IDs) until the row is resolved.
5. The healer must handle two rows resolving to the same page. After backfill, two legacy rows can map to one key, which the new unique index forbids. The healer keeps the first and reports the other as a duplicate instead of failing.
6. A newer file imports into an older build. Because unknown properties are ignored, the added fields are simply dropped. This holds only while no import code turns on strict member handling, so it is called out as a constraint.
7. Enrichment (decided). Export adds title, created, notebookKey, and sectionKey to each favorite and layout window, plus a small formatVersion field at the top of the file. An import on another machine can then resolve against the local hierarchy by title and creation time instead of falling back to location and title alone. It is additive and compatible with every point above: old builds ignore the new fields, and the formatVersion lets later changes detect which shape they are reading. pageKey is still never exported.

 

 

7. Migration and versioning (summary)

 

Each catalog keeps its own version table and upgrade chain. The implementation plan will give the details; this is the shape.

 

 

Principles:

 

1. Schema changes are pure SQL inside UpgradeCatalog, one transaction per step, ending with the version update, so a failed step leaves the old version in place. Providers are constructed in many contexts (dialogs, the tray, the calendar app) and must never need OneNote to open.
2. Data that needs OneNote is deferred. Resolving legacy favorites and layout windows needs the hierarchy, so it is done by the healer when OneNote is available, not during the upgrade. Until then, rows behave exactly as they do today.
3. Fresh databases get the final shape from the embedded DDL at the current version; existing databases get the upgrade chain. Both paths must produce the same schema and are tested against each other.
4. A database newer than the code is left alone. If a catalog reports a version the code does not know, the provider logs it and runs no step, rather than guessing. (A developer database can hold a newer version from another build.)
5. Shared helpers. ColumnExists and an upsert-style UpgradeSchemaVersion are currently private to individual providers and are lifted into DatabaseProvider. The upsert form is required because a version row may not exist yet (as with layouts_schema).
6. DDL stays single-line per statement with the CREATE ... IF NOT EXISTS name form, because RefreshDataSchema and DropCatalog parse it that way.

 

 

8. Failure modes and open questions

 

Known limits

 

 

 

Decisions

 

 

Open questions

 

These are measurements and a deferral, not preferences. They need OneNote to answer, or the work that creates the need.

 

1. Cadence and cost on large notebooks. The largest notebook measured had 447 live pages. The old scanner capped new notebooks at 100 pages, apparently to protect OneNote's responsiveness. The hierarchy-only pass needs to be timed on a much larger notebook, and a cheap skip (for example, comparing a notebook's lastModifiedTime) should be evaluated. This sets the default cycle length.
2. Does the pages-scope hierarchy include page-level metadata? The hashtag scanner skips its own tag-index pages by reading a page meta element obtained through GetSection. If the notebook-wide pages scope does not carry it, either section calls remain or another way to skip those pages is needed.
3. Paragraph identity is unmeasured. Anything that links to a specific paragraph needs its own test before it can build on this layer. This matters most for the later text index, whose results navigate to a paragraph by its object ID.

 

Not measured

Paragraph object IDs across a reopen, a second machine, renames of notebooks, sections, or pages, moving or renaming section groups, locked sections, the OneNote UI as the route for moves and date edits (the experiments used the API), other platforms, and very large notebooks. See the technote.

 

 

9. Testing strategy

 

 

The scan loop and the commands talk to OneNote directly and cannot be unit tested; the design keeps logic out of them so that what is left untested is thin.

 

 

═══════════════════════════════════════════════════════════════════════════════════════════════════

 

Architecture PlantUML (Refresh)

@startuml

skinparam componentStyle rectangle

skinparam shadowing false

package "OneNote" {

component "Hierarchy" as H

}

package "OneMore add-in" {

component "IdentityService" as IS

component "PageIdentityMatcher" as M

component "Identity resolver" as R

component "Hashtag stage" as HS

component "Workspace healer" as WH

component "FavoritesCommand,\nFavoritesMenu" as FC

component "RestoreLayoutCommand" as RL

component "SaveLayoutCommand,\nAddFavoriteCommand" as SV

}

package "OneMore.db" {

database "identity_page" as IDP

database "hashtag tables" as HT

database "favorite" as FAV

database "layout tables" as LAY

}

H --> IS : pages + attributes

IS --> M : PageRefs vs stored rows

M --> IS : resolutions

IS --> IDP : reconcile, soft-missing, purge

IS ..> HS : after each pass

IS ..> WH : after each pass

HS --> HT

WH --> FAV

WH --> LAY

FC --> R

RL --> R

SV --> R : register page at creation

R --> IDP

FC --> FAV

RL --> LAY

@enduml

 

Data Model PlantUML (Refresh)

@startuml

hide circle

skinparam linetype ortho

entity identity_page {

* pageKey : INTEGER <<PK, AUTOINCREMENT>>

--

pageID : TEXT (last seen, session-scoped)

notebookKey : TEXT

sectionKey : TEXT

title : TEXT

created : TEXT

modified : TEXT

level : INTEGER

missingSince : TEXT (NULL = present)

lastSeen : TEXT

}

entity hashtag_page {

* moreID : TEXT (holds the page key)

--

pageID : TEXT (refreshed each pass)

notebookID : TEXT (refreshed)

sectionID : TEXT (refreshed)

path, name, titleID

}

entity hashtag {

* tag, objectID

--

moreID : TEXT (page key)

}

entity favorite {

* favoriteID : INTEGER

--

pageKey : INTEGER (new, NULL for containers)

notebookKey : TEXT (new)

sectionKey : TEXT (new; group or section path)

kind : TEXT

notebookID, sectionID, pageID, uri (last known)

name, alias, location, folderID, sortOrder

}

entity layout_window {

* windowID : INTEGER

--

pageKey : INTEGER (new)

notebookID, sectionID, pageID, uri (last known)

name, alias, location, zOrder, device, bounds

}

identity_page ||--o{ hashtag_page : moreID = pageKey

hashtag_page ||--o{ hashtag : moreID

identity_page ||--o{ favorite : pageKey

identity_page ||--o{ layout_window : pageKey

@enduml

 

Background Identity Pass PlantUML (Refresh)

@startuml

participant "IdentityService" as S

participant "OneNote" as O

participant "Matcher" as M

database "identity_page" as DB

S -> O : GetNotebooks()

loop each open notebook

S -> O : GetNotebook(id, pages)

O --> S : sections, pages (ID, name, created, modified)

S -> S : build PageRef list;\nnote locked sections as skipped

end

S -> DB : read candidate rows\n(scoped notebooks + missing rows)

S -> M : Match(rows, pages)

M --> S : resolutions + orphans

S -> DB : insert new, update changed,\nmark orphans missing

S -> DB : purge missing longer than grace

S -> S : notify hashtag stage and healer

@enduml

 

Notebook Reopened PlantUML (Refresh)

@startuml

actor User

participant "IdentityService" as S

database "identity_page" as DB

User -> User : close and reopen notebook\n(all IDs regenerated)

S -> DB : pass 1: only part of the notebook is listed

note right of DB

Listed pages are rehomed by fingerprint:

same key, new page ID.

Pages not yet listed are marked missing,

NOT deleted.

end note

S -> DB : pass 2: more pages listed, more rehomed

S -> DB : pass N: whole notebook listed

note right of DB

Every page has its original key.

missingSince is cleared.

Nothing was lost.

end note

@enduml

 

Navigating Favorite PlantUML (Refresh)

@startuml

actor User

participant "FavoritesCommand" as C

participant "Resolver" as R

database "identity_page" as DB

participant "OneNote" as O

database "favorite" as F

User -> C : click favorite

C -> R : Resolve(favorite.pageKey)

R -> DB : read row

DB --> R : current pageID, section, notebook

R -> O : GetHyperlink(pageID)

O --> R : uri

alt resolved and navigation succeeds

C -> O : NavigateTo(uri)

C -> F : persist refreshed IDs and uri\n(if they changed)

else stale or unresolved

R -> O : read just that notebook's hierarchy

R -> R : reconcile that notebook, retry

alt now resolved

C -> O : NavigateTo(uri)

C -> F : persist refreshed IDs and uri

else still unresolved

C -> User : mark favorite as broken\n(never deleted)

end

end

@enduml

 

Restoring a Layout PlantUML (Refresh)

@startuml

participant "RestoreLayoutCommand" as C

participant "Resolver" as R

participant "OneNote" as O

loop each layout_window

C -> R : Resolve(window.pageKey)

R --> C : current pageID + uri (or unresolved)

alt resolved

C -> O : find open window for resolved pageID

alt not already open

C -> O : NavigateTo(resolved uri, newWindow)

C -> O : wait for window with resolved pageID

end

C -> O : position window

else unresolved

C -> C : record as skipped and report it\n(instead of silently ignoring)

end

end

@enduml

 

Adding a Favorite PlantUML (Refresh)

@startuml

participant "AddFavoriteCommand /\nSaveLayoutCommand" as C

participant "OneNote" as O

participant "Resolver" as R

database "identity_page" as DB

C -> O : read current page\n(title, created, section path, notebook path)

C -> R : EnsurePage(pageRef)

R -> DB : find by ID, else reconcile, else insert

R --> C : pageKey

C -> C : store pageKey + last-known IDs and uri

@enduml

 

Background Healing PlantUML (Refresh)

@startuml

participant "Workspace healer" as W

database "identity_page" as ID

database "favorite / layout_window" as T

participant "OneNote" as O

W -> T : rows needing attention

note right of T

1. pageKey is NULL (legacy rows)

2. stored page ID differs from

the identity's current ID

end note

alt legacy row (no pageKey)

W -> ID : match stored IDs against current rows

alt no match

W -> O : resolve by location path + title

end

W -> T : set pageKey, refresh IDs, location

else known key, stale IDs

W -> ID : current IDs

end

W -> O : GetHyperlink(page) to regenerate uri

W -> T : persist IDs and uri

@enduml

 

 

#omwiki #omdeveloper #omdesign

 

© 2026 Steven M Cohn. All rights reserved.

Please consider a sponsorship or one-time donation to support ongoing development